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Abstract — We consider a Gelfand-Pinsker discrete memoryless 
channel (DMC) model and provide a strong converse for its 
capacity. The strong converse is then used to obtain an upper 
bound on the reliability function. Instrumental in our proofs is a 
new technical lemma which provides an upper bound for the rate 
of codes with codewords that are conditionally typical over large 
message dependent subsets of a typical set of state sequences. This 
technical result is a nonstraightforward analog of a known result 
for a DMC without states that provides an upper bound on the 
rate of a good code with codewords of a fixed type (to be found 
in, for instance, the Csiszar-KSrner book). 

I. Introduction 

We consider a state dependent discrete memoryless channel 
(DMC), in which the underlying state process is indepen- 
dent and identically distributed (i.i.d.) with known probability 
mass function (pmf). The transmitter is provided access at 
the outset to the entire state sequence prevailing during the 
transmission of a codeword. The capacity of this DMC with 
noncausal channel state information (CSI) at the transmitter 
was determined in JT). Known popularly as the Gelfand- 
Pinsker channel, it has been widely studied for a broad range 
of applications which include fingerprinting, watermarking, 
broadcast communication, etc. 

In this paper, we are concerned with the strong converse for 
this channel as well as its reliability function, i.e., the largest 
exponential rate of decay, with block codeword length, of the 
decoding error probability. Even for a DMC without states, the 
reliability function is not fully characterized for all rates below 
channel capacity. Our main contributions are the following. 
First, we provide a strong converse for the capacity of the 
Gelfand-Pinsker DMC model, that is of independent interest. 
Second, using this strong converse, we obtain an upper bound 
for the reliability function; the later constitutes a line of attack 
described earlier (see, for instance, J2)). Instrumental in the 
proofs of both is a new technical result which provides an 
upper bound on the rate of codes with codewords that are 
conditionally typical over large message dependent subsets 
of a typical set of state sequences. This technical result is 
a nonstraightforward analog of J21 Lemma 2.1.4] for a DMC 
without states; the latter provides a bound on the rate of a 
good code with codewords of a fixed type. 



II. Preliminaries 

Consider a state dependent DMC W : X x S — > y 
with finite input, state and output alphabets X, S and y, 
respectively. The 5-valued state process {St}^2. 1 is i.i.d. with 
known pmf ?$■ The probability law of the DMC is specified 
by 

n 

W n (y\ X ,s)=l[w(y t \x t ,s t ) : 

t=i 

xe X n ,seS n , yey n . 

We consider the Gelfand-Pinsker model (Tj in which the 
encoder possesses perfect CSI in a noncausal manner, i.e., the 
entire state sequence prior to transmission. A (M, n)-code is 
a pair of mappings (/, <f>) where the encoder / is a mapping 

/ : M X S n -> X n 

with M = {1, . . . , M} being the set of messages, while the 
decoder is a mapping 

4> y n -> M. 

The rate of the code is (l/n)logM. The corresponding 
(maximum) probability of error is 

e(f,4>) = max V P s (s)x 
mEM * — ' 

W n ((^ 1 (m)) c I /(m,s),s) (1) 

where _1 (m) = {y € y n : 0(y) = m} and (-) c denotes 
complement. 

We restrict ourselves to the situation where the receiver has 
no CSI. When the receiver, too, has (full) CSI, our results 
apply in a standard manner by considering an associated DMC 
with augmented output alphabet y x S. 

Definition 1. Given < e < 1, a number R > is e- 
achievable if for every 5 > and for all n sufficiently large, 
there exist (M, n)-codes (/, </>) with (1/n) log M > R — 5 and 
e(/, 4>) < e; R is an achievable rate if it is e-achievable for 
all < e < 1. The supremum of all achievable rates is the 
capacity C of DMC. 



For a random variable U with values in a finite set U, let Furthermore, let (/, <ft) satisfy one of the following two con- 
V denote the set of all pmfs Pjjsxy on UxSxXxy with ditions 



X = h(U,S) 



for some mapping h, 



U -^S,X -o-Y, 
Py\x,s = W. 



(2) 



(3) 
(4) 



As is well-known H) 



C = max/(J7 A Y) - I(U A S). 
When the receiver, too, has (full) CSI it is known J5] that 

C = maxI(X AY I S). 

Px\s 

Definition 2. The reliability function E(R), R > 0, of the 
DMC W with noncausal CSI, is the largest number E > 
such that for every 5 > and for all sufficiently large n, there 
exist n-length block codes (/, <f>) as above of rate greater than 
R — S and e(/, <f>) < exp [—n(E — 5)} (see for instance J2])- 

For a given pmf P g x on 5 x ^, denote by V(Psx, W) the 
subset of V with P sx = Psx- 

III. Statement of Results 

An upper bound for the reliability function E(R), < R < 
C, of a DMC without states, is derived in J2] using a strong 
converse for codes with codewords of a fixed type. For a state 
dependent DMC with causal CSI at the transmitter and no 
receiver CSI, a strong converse is given in 0. An analogous 
result is not available for the case of noncausal transmitter 
CSI. For the latter situation, the following key lemma serves, 
in effect, as an analog of J2] Corollary 2.1.4] and gives an 
upper bound on the rate of codes with codewords that are 
conditionally typical over large message dependent subsets 
of the typical set of state sequences. We note that a direct 
extension of |2] Corollary 2.1.4] would have entailed a claim 
over a subset of typical state sequences not depending on the 
transmitted message; however, its validity is unclear. 

For a DMC without states, the result in Corollary 2.1.4] 
provides, in effect, an image size characterization of a good 
codeword set; this does not involve any auxiliary rv. In the 
same spirit, our key technical lemma below provides an image 
size characterization for good codeword sets for the noncausal 
DMC model, which now involves an auxiliary rv. 

Lemma 1. Let e, t > be such that e + r < 1. Given a pmf 
?s on S and conditional pmf Px\s> let {f,<j)) be a (M,n)- 
code as above. For each m G M., let A{m) be a subset of S n 
which satisfies the following conditions 



\\A(m) 



> 



exp 



A(m) C 7p" 



/(m,s)e7jg x] (s), seA(m). 



(5) 
(6) 
(7) 



W r ' 
1 



\A(m) 



4> 1 (m) | /(to, s),s) > 1 - e, ssi(m), (8a) 
■ ]T W n (0- 1 (m)|/(m J s),s) 

seA(m) 

> 1 - e. (8b) 
Then,fo£\ n > N(\\X\\, \\S\\, \\y\\,T,e), it holds that 

- log M < I(U AY) - I(U A S) 
n 

where P US xy £ V(P s Px\s, W). 

This lemma plays an instrumental role in proving the 
following two main results. 

Theorem 2. (Strong Converse) Given < e < 1 and a 
sequence of (M„,n) codes (f n ,<p n ) with e{f ni <f> n ) < e, it 
holds that 

lim sup — log M n < C. 

n n 



Theorem 3. (Sphere Packing Bound) Given 5 > 0, for < 
R < C, it holds that 



where 
Esp 

with 



E(R) <E SP {1 + S) + S, 

[D(Ps\\P S ) 



mm max mm 

Ps Pxis VeV(R,P s P x \s) 



(9) 



D(V\\W | P s P x \s)] 



V{R,P S x) = {V: XxS^y 



max I(U AY) - I(U A S) < R}. 

Pusxy&V(P S x,V) 

Remark 1. For the case when the receiver, too, possesses 
(full) CSI, the sphere packing bound above coincides with 
that obtained earlier in H for this case. 

Remark 2. In ©, the terms D(P g ||P 5 ) and D(V||W | 
PsPx is) account, respectively, for the shortcomings of a given 
code for corresponding "bad" state pmf and "bad" channel. 

IV. Proofs of Results 

We provide below the proofs of Lemma Q] and Theorems [2] 
and [3] 

Proof of Lemma [7} 
Our proof below is for the case when d8at holds; the 
case when dFbl holds can be proved similarly with minor 
modifications. Specifically, in the latter case, we can find 

'in our assertions, we indicate the validity of a statement "for all n > N(.)" 
by showing the explicit dependency of N; however the standard picking of 
the "largest such N" from (finitely-many) such TVs is not indicated. 



subsets A'(m) of A(m), m G M, that satisfy ©-(O and 
dial ) for some e',r'0 with e' + r' < 1 for all n sufficiently 
large. 
Set 

B{m) = {(/(m,s),s) G X n x5":se m G M. 

Let Py = Psx o W be a pmf on y defined by 

?Y{y)=^?sx{s,x)W{y\x,s), y ey. 

s,x 

Consequently, 

W n (Tg Y] | /(m,s),s) > e + r, s G A(m), (10) 

for all n > iV(||^||, |<S||, |3^||,T, e) (not depending on m and 
s in A(m)). Denoting 

C( m ) = f 1 ( m )n7| y]1 

we see from dHaJi and ( TTOb that 

W n (C(m) | f(m, s), s) > r > 0, (/(m, s), s) G B(m), 
so that 

\\C(m)\\ >g wn (B(m),T), 

where gn/»(B(m),r) denotes the smallest cardinality of a 
subset D of with 

W n (D | (/(m,s),s)) > t, (f(m, s),s) G B(m). (11) 

With to = argmin!< m < fl / ||C(to)[|, we have 



M||C(m )|| < E 11^)11 = ll^]ll <cx P n if(P y ) + 



Tn— 1 



G 



Consequently, 



ilogM < ff(P y ) + T - - - logg W n(B(TO ),T). (12) 
n b n 

Define a stochastic matrix V : A" x S — > <S with 

V( fl / |x, a ) = l(a' = s) J 

and let g V n be defined in a manner analogous to g w « above 
with S n in the role of y n in dTTJ. For any m£M and subset 
E of 5™, observe that 

| /(to, s), s) = l(s G E), s G 5™. 

In particular, if E satisfies 

V n (E | /(to, s), s) > t, s g A(m), (13) 

it must be that A(m) C and since E = A(m) satisfies 
( flJl l, we get that 



P(TO)||=g v »(S(TO),r) 



using the definition of B{m). Using the image size character- 
ization |2] Theorem 3.3.11], there exists an auxiliary rv U and 
associated pmf Pusxy = ^u\sx^SxW such that 



-log Sv u(B(m ),T)-H(S\U)-t 
n 

-loggw(B(m ),T) - H(Y\U) -t 



< 



6' 



T 

<6' 



(15) 



where < t < mm{I(U A Y), I(U A S)}. Then, using (fT2l) . 
( fT4b , (fT3T > we get 



1 



logM < I(U AY) + H(S | [/) 



-log||A(m )|| + J, 
n 2 



which by (|6]i yields 
1 



log M < I(U AY)- I(U A S) + t. 

In ( fTBT ), Pc/sxf belongs to V(PsPx\s, W) but need not 
satisfy ([2j. Finally, the asserted restriction to Pc/sxy G 
^(PsPxis, W) follows from the convexity of I(U AY) - 
I(U A S) in ?x\us f° r a nxe d Ptrs (as observed in (T|). ■ 

Proof of Theorem [2} 
Given < e < 1 and a (M, n)-code (/, 0) with e(/, 4>) < e, 
the proof involves the identification of sets A(m), to G M, 
satisfying (JSj-dVj and (ISal l. The assertion then follows from 
Lemma Q] Note that e (/,</>) < e implies 



2 p s ( s ) W 



n,J -- 1 (m) | /(to,s),s) > 1-e 

s65" 

for all to G M. Since P s ( 7^. 



for every meM 
Ps 



s G Tfo : 



1 as n — > oo, we get that 

1-e' 



_1 (m) | /(to,s),s) > 



> 



1 - e 



(16) 



for all n > iV(||<S||, e). Denoting the set {•} in (O by A(m) 
clearly for every to G M, 

1 - , 



and 



W n (4>- 1 (m) | /(to,s),s) > 
Ps (Mm)) > 



2 ' 

1 - 6 

3 



s G A(m) 



(14) 



for n > iV(||5||, e), whereby for an arbitrary 5 > 0, we get 

\\A(m)\\ >exp [n(ff(P s ) - *)] 

for n > iV(||«S||, 5). Partitioning A(m), m G AL into sets 
according to the (polynomially many) conditional types of 
/(m,s) given s in A(m), we obtain a subset A(m) of A(m) 
for which 

/(to, s) G 7^J(s), sgA(to), 
||A(m)|| >exp[n(£T(Ps)-2<f)], 



for n > iV(||«S||, \\X\\, 6), where ^(s) represents a set of 
those sequences in X n that have the same conditional type 
(depending only on m). 

Once again, the polynomial size of such conditional types 
yields a subset M' of M. such that /(m,s) has a fixed 
conditional type (not depending on m) given s in A(m), and 
with 

-loglLM'H > - logM-6 
n n 

for all n > N(\\S\\,\\X\\,5). Finally, the strong converse 
follows by applying Lemma Q] to the subcode corresponding 
to M.' and noting that 5 > is arbitrary. ■ 

Proof of Theorem \3} 
Consider sequences of type Ps in S n . Picking A(m) = 
T- 1 , m £ M, in the proof of Theorem [2] and following the 
arguments therein to extract the subset A(m) of A(m), we 
have for a given S > that for n > iV(||5||, H^H,^), there 
exists a subset A4' of Ai and a fixed conditional type, say 
Px\s ( not depending on m), such that for every m G Ad', 



rr-n 



A(m) C A{m) 
\\A(m)\\>exp[n(H(P s ) 
/(m,s)e7p" |s (s), 

-log||A4'|| > R-S. 



-S)}, 
s G A(m) 



Then for every V £ V(R,~PsPx\s)> we obtain using Lemma 
Q] (in its version with condition d8bl), that for every 5' > 0, 
there exists m £ M 1 (possibly depending on 5' and V) with 

E V»r(m)n/K«),«)>l-f 

for all 7i > ^(11511,11^11,11^11,(5'). For this m, apply (2] 
Theorem 2.5.3, (5.21)] with the choices 



Z 

s 

Qi(y,s) 



y n x A(m), 
(^ _1 (m)) c x A(m), 
P(y|/(m,8),s) 



Q2(y,s) = 

for (y, s) G Z, to obtain 



W n (y | /(m,s),s) 



raxr (( '" 1(m))0|/(m, ' ),B) 



> exp 



ng(y||^ I P X |sPg) + 1 
1-5' 



Finally, 

e(/»> E P s (s)M/"((r 1 (m)) c |/(m,s),s) 

sgA(m) 

> exp[-n(D(P s \\Ps) 

+ D(V\\W\P XIS P S )(1 + S) + S)} 

for n > ^(||5||, \\y\\, S, 5'), whereby it follows that 

limsup- -loge(/,(/j) 
n n 

< minmax min [_D(Ps||Ps) 

Ps Px|S V£V(R,PsPx\s) 

+ .0(^11^ | P X | S P S )(l + 5)+»5] 

for every 6 > 0. 
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